The Role of Semantic Processing in an Automatic Speech Understanding System

We present the semantics component of a speech understanding and dialogue system that is developed at our institute. Due to pronunciation variabilities and vagueness of the word recognition process, semantics in a speech understanding system has to resolve additional problems. Its main task is not only to build up a representation structure for the meaning of an utterance, as in a system for written input, semantic knowledge is also employed to decide between alternative word hypotheses, to judge the plausibility of syntactic structures, and to guide the word recognition process by expectations resulting from partial analyses. 1. I n t r o d u c t i o n Understanding spoken utterances requires more than mere word recognition. It is based on a number of meaning aspects, covering the range from textual interpretation of a sentence up to the revelation of the speaker's intention in the context of a special dialogue situation. In the speech understanding and dialogue system EVAR /4 / , task-independent semantic analysis, domain-dependent pragmatic analysis, and dialogue-specific aspects are implemented in three separate modules /2/. Semantic analysis comprises those aspects that can be studied at the isolated sentence, independent from its actual use in the dialogue. The semantics module disregards communicative aspects of an utterance as well as its situational and thematic context. Thus, semantic consistency of words and constituents and underlying relational structure of the sentence are the main points of interest in this stage of analysis. Semantic knowledge consists of lexical meanings of words and selectional restrictions between them. The analysis of the functional structure is based on the principles of case and valency theory. 2. V a l e n c i e s a n d c a s e t h e o r y The theoretical background for the analysis of functional relations in a sentence is given by valency and case theory /5, 3/. The main idea is that the syntactic and the semantic structure of a sentence are essentially determined by its head verb. The property to call for a certain number and kind of complementary noun groups or prepositional groups that are necessary to build up an adequate sentence is called valency. The morpho-syntactic and semantic descriptions of the complements constitute a verb frame with slots to be filled up by actual phrases. This valency frame is augmented by case labels circumscribing the functional role of the expected phrase. To give an example, the verb "suchen" (to look for) has the case slots: AGENT: noun group (nominative), ANIMATE, obligatory OBJECT: noun group (accusative), obligatory LOCATION: adverbial group, PLACE, facultative. The lexical knowledge base provides caseframe entries for all verbal and nominal items with valency properties. Mostly, meaning alternatives correspond to different caseframes. We use a relatively detailed case system with about 30 cases. For use within the semantics module, a preprocessor transforms the dictionary entries to a network representation of concepts. The network scheme is influenced by the formalism of Structured Inheritance Networks /1/ and is described in /2/ . It is used for knowledge representation in all semantic and pragmatic modules in the system. Similar to the frame theoretic approach, the underlying assumption in case theory is that words evoke certain contextual expectations to the hearer, based on his personal experiences and his knowledge on stereotypic situations. In our system, this assumption is adopted in that we use case descriptions not only for verifying syntactic hypotheses, but also for syntactic and semantic predictions about the rest of the sentence. Tiffs topdown aspect plays an essential role not only in the semantic component but in the whole recognition process. 3. Semantic r e a s o n i n g in EVAR In our speech understanding system, the semantic analysis as defined above comprises the following tasks: resolution of lexical ambiguities interpretation of constituents with respect to their semantic features choice between alternative syntactic hypotheses and between alternative interpretations of constituents revelation of semantic anomalies due to recognition errors representation of the case structure inference of expectations on the rest of the sentence. These problems are solved by three fundamental operations of the semantics module: local interpretation by unification of semantic features, contextual interpretation by case frame analysis, and top-down hypotheses. 3.1 Local interpretation of constituents One of the main tasks of the module consists in mapping syntactic structures (hypotheses) to caseframe instances. As this mapping essentially relies on semantic features, the features of a phrase have to be determined first. On the one hand, this means resolution of lexical ambiguities, on the other hand, this process supports the choice between alternative word and structural hypotheses. The principle is to reduce lexical ambiguities by selectionaI features of the phrase heads that constrain dependent words and phrases. To determine the features of a phrase, all meaning alternatives of its constituents are unified and tested for compatibility. The test yields a rating that is the higher, the more constituents are compatible with the nucleus class. Of all possible feature combinations, the one with the highest consistency is chosen. The semantic consistency rating of a group can also be regarded as a measure for the plausibility of a syntactic hypothesis. As low semantic ratings may result from grouping wrong word hypotheses, a search for alternative word and constituent hypotheses may be reasonable in an area with bad semantic consistency. The combinatoric constraints of words are expressed in the dictionary by the feature SELECTION. The system of semantic classes (features) is organized in a conceptual hierarchy, thus, with a given class selected by the phrase head all its subclasses are accepted as compatible. The system presently used consists of about 110 semantic features and is represented as a concept hierarchy in the network formalism.